Multicentric development and validation of a multi-scale and multi-task deep learning model for comprehensive lower extremity alignment analysis.

Osteoarthritis of the knee, a widespread cause of knee disability, is commonly treated in orthopedics due to its rising prevalence. Lower extremity misalignment, pivotal in knee injury etiology and management, necessitates comprehensive mechanical alignment evaluation via frequently-requested weight-bearing long leg radiographs (LLR). Despite LLR's routine use, current analysis techniques are error-prone and time-consuming. To address this, we conducted a multicentric study to develop and validate a deep learning (DL) model for fully automated leg alignment assessment on anterior-posterior LLR, targeting enhanced reliability and efficiency. The DL model, developed using 594 patients' LLR and a 60%/10%/30% data split for training, validation, and testing, executed alignment analyses via a multi-step process, employing a detection network and nine specialized networks. It was designed to assess all vital anatomical and mechanical parameters for standard clinical leg deformity analysis and preoperative planning. Accuracy, reliability, and assessment duration were compared with three specialized orthopedic surgeons across two distinct institutional datasets (136 and 143 radiographs). The algorithm exhibited equivalent performance to the surgeons in terms of alignment accuracy (DL: 0.21 ± 0.18°to 1.06 ± 1.3°vs. OS: 0.21 ± 0.16°to 1.72 ± 1.96°), interrater reliability (ICC DL: 0.90 ± 0.05 to 1.0 ± 0.0 vs. ICC OS: 0.90 ± 0.03 to 1.0 ± 0.0), and clinically acceptable accuracy (DL: 53.9%-100% vs OS 30.8%-100%). Further, automated analysis significantly reduced analysis time compared to manual annotation (DL: 22 ± 0.6 s vs. OS; 101.7 ± 7 s, p ≤ 0.01). By demonstrating that our algorithm not only matches the precision of expert surgeons but also significantly outpaces them in both speed and consistency of measurements, our research underscores a pivotal advancement in harnessing AI to enhance clinical efficiency and decision-making in orthopaedics.

Toward telemedical diagnostics-clinical evaluation of a robotic examination system for emergency patients.

Introduction: The SARS-CoV-2 pandemic has affected global public healthcare for several years. Numerous medical professionals have been infected since the outbreak in 2019, resulting in a shortage of healthcare providers. Since traditional personal protective wear was insufficient to eliminate the virus transmission reliably, new strategies to avoid cross-infection were imperative while enabling high-quality medical care. In the project ProteCT, we investigated the potential of robotic-assisted examination in providing medical examination via a telemedical approach. Material and Methods: We constructed a fully functional examination cabin equipped with cameras, microphones, screens and robotic arms to evaluate usability and perception. Therefore, we conducted a preliminary study with 10 healthy volunteers and 10 physicians to gain first insights and optimize the setup. In a second step, we performed telemedical examinations of actual patients from the local emergency department to compare the robotic approach with the classical method of measuring vital signs, auscultation, palpation and percussion. Results: The preliminary study identified basic requirements, such as the need for force-feedback and telemedical training for physicians. In the main study, acceptance was high and most patients indicated they would use a telemedical system again. Our setup enabled the physician to make the same diagnoses as by classic examination in the emergency department in most cases. Discussion: The potential acceptance of a telemedical system such as ProteCT is high. Robotic telemedical approaches could complement future healthcare beyond the Corona pandemic to reach rural areas or even war zones. Moreover, the daily clinical use of robotic telemedicine could improve patients' safety, the quality of perioperative management and the workflow in any medical facility. Conclusion: The development of telemedical and telerobotic systems is a multidisciplinary and complex challenge. However, acceptance of the proposed system was high among patients and physicians, indicating the potential use of similar systems for future healthcare.

Autonomous swab robot for naso- and oropharyngeal COVID-19 screening.

The COVID-19 outbreak has triggered a global health and economic crisis, necessitating widespread testing to control viral spread amidst rising cases and fatalities. The recommended testing method, a combined naso- and oropharyngeal swab, poses risks and demands limited protective gear. In response to the COVID-19 pandemic, we developed and tested the first autonomous swab robot station for Naso- and Oropharyngeal Coronavirus Screening (SR-NOCS). A force-sensitive robot running under a Cartesian impedance controller is employed to drive the swab to the sampling area. This groundbreaking device underwent two clinical studies-one conducted during the initial pandemic lockdown in Europe (early 2021) and the other, more recently, in a public place after the pandemic had subsided earlier in the year 2023. In total, 52 patients suspected of COVID-19 infection were included in these clinical studies. The results revealed a complete positive correlation between autonomous and manual sampling. The test subjects exhibited a high acceptance rate, all expressing a willingness to undergo future tests with SR-NOCS. Based on our findings, such systems could enhance testing capabilities, potentially conducting up to 300 tests per robot per day with consistent precision. The tests can be carried out with minimal supervision, reducing infection risks and effectively safeguarding patients and healthcare workers.

From Human Hand to Grasp Surface Detection, Tracking & Analysis.

Hands are paramount for dexterous interactions that humans exhibit in daily life. Understanding the intricacies of human hand-object interactions is therefore necessary. Unfortunately, the limitations of state-of-the-art technologies make capturing the full hand-object complexity unfeasible, giving rise to the need for new technological means to achieve this aim. In this work, we propose an end-to-end framework in which individualized hand models are derived and used to capture quantitative personalized hand-object interaction information, precisely, hand shape, kinematics, and contact surfaces. The results of this study serve as a proof of concept that such a framework can significantly deepen personalized hand-object interaction analyses, providing, in perspective, insights for medical diagnoses and rehabilitation, among others.Clinical relevance- Our work showcases the need to incorporate bespoke human hand models in individualized hand function assessment technologies, as hand-object interaction information is subject-dependent.

Robust Independent Component Analysis based EMG decomposition - a comparison study.

High density surface Electromyography (HD-sEMG) provides a high fidelity measurement of the myoelectric activity that can be leveraged by EMG decomposition methods to estimate the motor neuron discharges. Independent Component Analysis (ICA) methods are used as basis for many EMG decomposition algorithms, for the estimation of motor unit action potential signals. Accurate source separation is a non-trivial task in EMG decomposition. While FastICA is widely used for this purpose, other methods with attractive characteristics, such as RobustICA, remain relatively unexplored. The purpose of the current work is to compare three different ICA-based EMG decomposition methods (FastICA, RobustICA and RobustICALCH) in terms of decomposition accuracy and computation time. The evaluation was performed on simulated data using a decomposition algorithm inspired by previous studies. Our results demonstrate that RobustICA outperforms the other methods in terms of number of correctly identified motor units, high decomposition accuracy, and low computation time, across different muscle contraction levels.

Machine learning-driven self-discovery of the robot body morphology.

The morphology of a robot is typically assumed to be known, and data from external measuring devices are used mainly for its kinematic calibration. In contrast, we take an agent-centric perspective and ponder the vaguely explored question of whether a robot could learn elements of its morphology by itself, relying on minimal prior knowledge and depending only on unorganized proprioceptive signals. To answer this question, we propose a mutual information-based representation of the relationships between the proprioceptive signals of a robot, which we call proprioceptive information graphs (π-graphs). Leveraging the fact that the information structure of the sensorimotor apparatus is dependent on the embodiment of the robot, we use the π-graph to look for pairwise signal relationships that reflect the underlying kinematic first-order principles applicable to the robot's structure. In our discussion, we show that analysis of the π-graph leads to the inference of two fundamental elements of the robot morphology: its mechanical topology and corresponding kinematic description, that is, the location and orientation of the robot's joints. Results from a robot manipulator, a hexapod, and a humanoid robot show that the correct topology and kinematic description can be effectively inferred from their π-graph either offline or online, regardless of the number of links and body configuration.

Real-Time-Capable Muscle Force Estimation for Monitoring Robotic Rehabilitation Therapy in the Intensive Care Unit.

In this paper, a method is proposed to enable real-time monitoring of muscle forces during robotic rehabilitation therapy in the ICU. This method is solely based on sensor information provided by the rehabilitation robot. In current clinical practice, monitoring primarily takes place in the later stages of rehabilitation, but it would also be highly beneficial during early stages. Musculoskeletal models have large, mostly unrealized potential to support and improve patient monitoring. The method presented in this paper is based on a state-of-the-art muscle-tendon path model, which is applied to the use case of the robotic rehabilitation device VEMOTION. The muscle force estimation is validated against surface electromyography measurements of lower limb muscles from 12 healthy volunteers The results show an overall correlation of R = 0.70 0.25 for the single-joint muscle m. iliopsoas, which has a ±major contribution to hip flexion. Given this correlation, the proposed model could be used for real-time monitoring of active patient participation.

Error-related Potentials in a Virtual Pick-and-Place Experiment: Toward Real-world Shared-control.

In Human-Robot Collaboration setting a robot may be controlled by a user directly or through a Brain-Computer Interface that detects user intention, and it may act as an autonomous agent. As such interaction increases in complexity, conflicts become inevitable. Goal conflicts can arise from different sources, for instance, interface mistakes - related to misinterpretation of human's intention - or errors of the autonomous system to address task and human's expectations. Such conflicts evoke different spontaneous responses in the human's brain, which could be used to regulate intrinsic task parameters and to improve system response to errors - leading to improved transparency, performance, and safety. To study the possibility of detecting interface and agent errors, we designed a virtual pick and place task with sequential human and robot responsibility and recorded the electroencephalography (EEG) activity of six participants. In the virtual environment, the robot received a command from the participants through a computer keyboard or it moved as autonomous agent. In both cases, artificial errors were defined to occur in 20% - 25% of the trials. We found differences in the responses to interface and agent errors. From the EEG data, correct trials, interface errors, and agent errors were truly predicted for 51.62% ± 9.99% (chance level 38.21%) of the pick movements and 46.84%±6.62% (chance level 36.99%) for the place movements in a pseudo-asynchronous fashion. Our study suggests that in a human-robot collaboration setting one may improve the future performance of a system with intention detection and autonomous modes. Specific examples could be Neural Interfaces that replace and restore motor functions.

Model-Based Shared Control of a Hybrid FES-Exoskeleton: An Application in Participant-Specific Robotic Rehabilitation.

Hybrid exoskeleton, comprising an exoskeleton interfaced with functional electrical stimulation (FES) technique, is conceptualized to complement the weakness of each other in automated neuro-rehabilitation of sensory-motor deficits. The externally actuating exoskeleton cannot directly influence neurophysiology of the patients, while FES is difficult to use in functional or goal-oriented tasks. The latter challenge is largely inherited from the fact that the dynamics of the muscular response to FES is complex, and it is highly user- and state-dependent. Due to the retardation of the muscular contraction response to the FES profile, furthermore, a commonly used model-free control scheme, such as PID control, suffers performance. The challenge in FES control is exacerbated especially in the presence of the actuation redundancy between the volitional activity of the user, powered exoskeleton, and FES-induced muscle contractions. This study therefore presents trajectory tracking performance of the hybrid exoskeleton in a novel model-based hybrid exoskeleton scheme which entices user-specific FES model-predictive control.

Assessing Human-Human Kinematics for the Implementation of Robot-Assisted Physical Therapy in Humanoids: A Pilot Study.

The development of humanoids with bimanual manipulator arms may facilitate assistive robots to perform physical therapy with older adults living at home. As we assume the human-human interaction to be the gold standard of physical therapy, we propose a kinematics analysis to derive guidelines for implementing physical therapy assisted by humanoids. Therefore, a pilot study was carried out involving three physical therapists and two participants acting as exemplary patients. The study analyzes the therapists' movement strategy, including the position and orientation of the therapists' bodies in relation to the participants and the placement of the therapists' hands on the upper limb segment of the participants, as well as the inter- and intravariability during the performance of a ROM (range of motion) assessment. The results demonstrate that while physical therapists exhibit variation in their interaction strategies, they still achieve a consistently low level of variability in their manipulation space.

Estimating Joint Kinematics and Muscles Forces During Robotic Rehabilitation to Detect and Counteract Reduced Ankle Mobility.

The paper presents a solution to detect active ankle joint movement while a patient undergoes therapy with a robotic lower limb rehabilitation device that neither restricts nor actively supports ankle dorsi- or plantarflexion. The presented method requires the addition of only two accelerometer sensors to the system as well as a musculoskeletal model of the lower limb. Using forward kinematics and inverse dynamics, it enables knee and ankle joint kinematic tracking in the sagittal plane and muscle force estimation. This is an extension of a previous work in which only hip joint tracking was possible and, thus, muscle force estimation was limited. The correlation results of the current validation study with 12 healthy subjects show high correlation (R=0.88±0.09) between the kinematics estimated with the proposed method and those calculated from a gold standard motion capture setup for all three joints (hip, knee, and ankle). The correlation results of the estimated m. tibialis anterior muscle force against electromyography measurements (R = 0.62±0.27) are promising and a first application to a patient data set shows potential for future clinical application.

Individualized Training of Back Muscles Using Iterative Learning Control of a Compliant Balance Board.

Here we present the GyroTrainer, a bespoke mechatronic balance board system designed to trigger activation of the back muscles while the user engages in a balance-challenging game. The GyroTrainer uses admittance control coupled with an iterative learning approach so as to tailor the admittance control parameters, i.e. difficulty level, according to the user's skill. Our experimental evaluation demonstrated that an individualized admittance control stiffness could be identified for each user, which corresponds with a desired level of difficulty and increased back muscle activity. A first game implementation demonstrates the feasibility of utilizing the GyroTrainer system and the individually identified admittance control stiffness for gamification of back muscle training.

Monitoring Active Patient Participation During Robotic Rehabilitation: Comparison Between a Robot-Based Metric and an EMG-Based Metric.

While rehabilitation robots present a much-needed solution to improving early mobilization therapy in demanding clinical settings, they also present new challenges and opportunities in patient monitoring. Aside from the fundamental challenge of quantifying a patient's voluntary contribution during robot-led therapy motion, many sensors cannot be used in clinical settings due to time and space limitations. In this paper, we present and compare two metrics for monitoring a patient's active participation in the motion. The two metrics, each derived from first principles, have the same biomechanical interpretability, i.e., active work by the patient during the robotic mobilization therapy, but are calculated in two different spaces (Cartesian vs. muscle space). Furthermore, the sensors used to quantify these two metrics are fully independent from each other and the associated measurements are unrelated. Specifically, the robot-based work metric utilizes robot-integrated force sensors, while the EMG-based work metric requires electrophysiological sensors. We then apply the two metrics to therapy performed using a clinically certified, commercially available robotic system and compare them against the specific instructions given to the healthy subjects as well as against each other. Both metric outputs qualitatively match the expected behavior of the healthy subjects. Additionally, strong correlations (median [Formula: see text]) are shown between the two metrics, not only for healthy subjects (n = 12) but also for patients (n = 2), providing solid evidence for their validity and translatability. Importantly, the robot-based work metric does not rely on any sensors outside of those integrated into the robot, thus making it ideal for application in clinical settings.

Validation of a Robotic Testbench for Evaluating Biomechanical Effects of Implant Rotation in Total Knee Arthroplasty on a Cadaveric Specimen.

In this study, we developed and validated a robotic testbench to investigate the biomechanical compatibility of three total knee arthroplasty (TKA) configurations under different loading conditions, including varus-valgus and internal-external loading across defined flexion angles. The testbench captured force-torque data, position, and quaternion information of the knee joint. A cadaver study was conducted, encompassing a native knee joint assessment and successive TKA testing, featuring femoral component rotations at -5°, 0°, and +5° relative to the transepicondylar axis of the femur. The native knee showed enhanced stability in varus-valgus loading, with the +5° external rotation TKA displaying the smallest deviation, indicating biomechanical compatibility. The robotic testbench consistently demonstrated high precision across all loading conditions. The findings demonstrated that the TKA configuration with a +5° external rotation displayed the minimal mean deviation under internal-external loading, indicating superior joint stability. These results contribute meaningful understanding regarding the influence of different TKA configurations on knee joint biomechanics, potentially influencing surgical planning and implant positioning. We are making the collected dataset available for further biomechanical model development and plan to explore the 6 Degrees of Freedom (DOF) robotic platform for additional biomechanical analysis. This study highlights the versatility and usefulness of the robotic testbench as an instrumental tool for expanding our understanding of knee joint biomechanics.

Interproximal tooth cleaning operated by a tactile robot. An in vitro analysis.

AIM: New technologies such as tactile robots and artificial intelligence are about to find their way into clinical practice in dentistry and may contribute to the improvement of oral health care in the future. In this study we hypothesized that a collaborative, tactile robot programmed by a dental student removes interproximal artificial plaque as effectively as a human operator in an in vitro pilot trial. MATERIAL AND METHODS: Model teeth were fully covered with artificial plaque and set into phantom jaws. First, a robot was programmed by a dental student to perform interproximal cleaning with an interproximal brush. Second, teeth were covered with artificial plaque again and the dental student performed the interproximal cleaning manually. Both experiments were repeated five times. Residual plaque was measured with binary pictures. Surface coverage was reported and comparison of methods was performed with significance defined at a= 0.05. RESULTS: No statistically significant difference was found in the cleaning result between the robot and the human operator. CONCLUSION: The results of this in vitro pilot study indicate that a tactile robot with integrated artificial intelligence programmed by a dental student can perform interproximal cleaning as effectively as the dental student. Practical lmplications: In the future, the use of robot assistants to support oral hygiene, e.g., in patients with reduced motor skills or impaired vision may be further investigated.

Dentronics: tooth cleaning with a tactile collaborative robot - an in vitro proof of concept.

AIM: The aim of the present study was to compare the performance of a tactile collaborative robot programmed by a dental professional (DP) with that of a DP in the removal of surrogate plaque in vitro. MATERIALS AND METHODS: Six typodont teeth in articulated jaws were covered with surrogate plaque and cleaned by a DP with the help of a manual toothbrush (DP/manual) and an electric toothbrush (DP/electric). The experiment was repeated with the help of a collaborative seven-axis tactile robot programmed by a DP handling a manual toothbrush (robot/manual) and an electric toothbrush (robot/electric). All experiments were repeated five times, resulting in a total of N = 30 teeth in each group. Cleaning results were reported as the percentage of surface area with residual plaque. RESULTS: The cleaning results of the DP and the robot showed no significant differences. However, electric toothbrushing was significantly less effective compared with manual toothbrushing (P < 0.05). CONCLUSION: The present in vitro study indicates that current robot technology may perform the removal of surrogate plaque as efficiently as a DP. In future, this may be helpful to release nursing staff from this time-demanding task that could possibly cause contagion or to support people with reduced motor skills or impaired vision in performing daily oral hygiene.

Tactile Robotic Telemedicine for Safe Remote Diagnostics in Times of Corona: System Design, Feasibility and Usability Study.

The current crisis surrounding the COVID-19 pandemic demonstrates the amount of responsibility and the workload on our healthcare system and, above all, on the medical staff around the world. In this work, we propose a promising approach to overcome this problem using robot-assisted telediagnostics, which allows medical experts to examine patients from distance. The designed telediagnostic system consists of two robotic arms. Each robot is located at the doctor and patient sites. Such a system enables the doctor to have a direct conversation via telepresence and to examine patients through robot-assisted inspection (guided tactile and audiovisual contact). The proposed bilateral teleoperation system is redundant in terms of teleoperation control algorithms and visual feedback. Specifically, we implemented two main control modes: joint-based and displacement-based teleoperation. The joint-based mode was implemented due to its high transparency and ease of mapping between Leader and Follower whereas the displacement-based is highly flexible in terms of relative pose mapping and null-space control. Tracking tests between Leader and Follower were conducted on our system using both wired and wireless connections. Moreover, our system was tested by seven medical doctors in two experiments. User studies demonstrated the system's usability and it was successfully validated by the medical experts.

Testing robot-based assist-as-needed therapy for improving active participation of a patient during early neurorehabilitation: a case study.

In this study, a patient in the Intensive Care-Unit received robot-based mobilization therapy with an assist-as-needed (AAN) function over the course of three weeks. Therapists were able to adapt the hip range of motion $\beta$, the bed verticalization angle $\alpha$ and the leg load force FLoad for each therapy, based on the current condition of the patient. To evaluate the patient active participation, surface electromyography (sEMG) of the M. rectus femoris (RF) and M. biceps femoris (BF) were measured and analyzed. It was observed that the patient active participation, measured through sEMG, increased along with increased hip range of motion $\beta$, bed verticalization angle $\alpha$ and leg load force FLoad set by the therapists. The patient muscle activation pattern followed the pattern of healthy controls, in part. To the authors' best knowledge, this study is the first of its kind to be performed with an ICU patient.

Integration of robotic in the reprocessing and transfer of endoscopes.

Background and study aims Optimal hygiene is crucial for patients undergoing flexible endoscopy. Reprocessing is currently influenced by manual procedures performed by endoscopy staff. To overcome this limitation, we designed and evaluated the integration of robotic application for an automated endoscope processing pathway. Methods We used an endoscope reprocessing pass through machine with drying cabinet and a Franka Emika Panda robot. The robot was programmed to interact with its environment in a compliant way, guaranteeing desired contact force thresholds and therefore ensuring safety of both robot and medical equipment. Results In an initial phase we tested the robots' ability to handle a modified tray holding an endoscope as well as certain challenges (correct positioning, connection of tubing, undesired collisions). We added another Panda robot arm resulting in a device featuring two independent manipulators and tested the accuracy of each individual step. We evaluated 50 consecutive processing and transfer procedures, simulating the average daily throughput of an endoscopic unit. The endoscopes were removed in adapted tray using a specially designed lifting device and placed in an endoscope storage and venting cabinet. The mean time for the handling of the scope was 104.2 ±â€Š1.2 seconds and an accuracy of 100 % (0 failures in 50 attempts) was achieved. Conclusions To the best of our knowledge, this is the first description and evaluation of an automated compliant robotic assistance in the processing of endoscopes. Further development could help to overcome shortcomings of the man handled endoscope processing and could lead to reproducible, standardized and certified endoscope processing.

Development of an Exoskeleton Platform of the Finger for Objective Patient Monitoring in Rehabilitation.

This paper presents the application of an adaptive exoskeleton for finger rehabilitation. The system consists of a force-controlled exoskeleton of the finger and wireless coupling to a mobile application for the rehabilitation of complex regional pain syndrome (CRPS) patients. The exoskeleton has sensors for motion detection and force control as well as a wireless communication module. The proposed mobile application allows to interactively control the exoskeleton, store collected patient-specific data, and motivate the patient for therapy by means of gamification. The exoskeleton was applied to three CRPS patients over a period of six weeks. We present the design of the exoskeleton, the mobile application with its game content, and the results of the performed preliminary patient study. The exoskeleton system showed good applicability; recorded data can be used for objective therapy evaluation.